1. Field of the Invention
The present invention relates to devices for monitoring around a vehicle. More specifically, the present invention relates to a device for monitoring around a vehicle in which target extraction is carried out by subjecting an image captured by an infrared camera device to a binary thresholding method.
2. Description of Related Art
Devices for monitoring around a vehicle have been proposed in which objects that may collide with the vehicle, such as pedestrians, are extracted from a picture around the vehicle captured by an infrared camera, and such information is supplied to a driver of the vehicle. In these devices, the probability that the vehicle will hit an object, such as a pedestrian, is calculated based on the relative distance between the vehicle and the object, and the relative speed of the vehicle.
An example of such devices for monitoring around a vehicle which extracts an object, which may collide with the vehicle, from an image around the vehicle captured by an infrared camera is as follows. That is, in the device, the captured infrared image is subjected to a binary (2-level) thresholding process, and an area to which bright (white) portions are concentrated is searched for. Then, it is determined whether the area is a head portion of a pedestrian by using the aspect ratio (ratio of length to width) and the sufficiency rate of the area, and further calculating the distance between the vehicle and the area using the actual surface area and the position of the center of gravity in the image. If the area of the head portion of a pedestrian is determined, an area which forms the body of the pedestrian is determined by calculating the height of the pedestrian in the image based on the distance between the area determined to be the head portion and the camera, and an average height of an adult. These areas are displayed to be distinguished from the other regions of the image. In this manner, the position of the entire body of the pedestrian in the infrared image is determined, and this information is displayed for the driver so as to effectively assist the vision of the driver (refer to Japanese Unexamined Patent Application, First Publication No. Hei 11-328364, for example).
However, using the thresholding method, only the head portion, a part of the head portion, or the entire or only an upper half or a lower half of the body, of a pedestrian may be extracted from an infrared image, depending on the effects of a hat or clothing the pedestrian is wearing, or of the environment surrounding the pedestrian, and thus the shape of the pedestrian obtained by the binary thresholding method becomes ambiguous. Also, when a vehicle is running, due to the influence in change in the shape of a road ahead, or the pitting of the vehicle, the height of a pedestrian, from a child to an adult, is generally detected to be different from his/her real height.
Accordingly, since the barycentric coordinates of targeted objects, such as pedestrians, in the picture cannot be fixed with respect to the distance, there is a possibility that the pedestrians cannot be stably extracted if the extraction is carried out based only on the shape of the pedestrians as in the above-mentioned conventional device.
Also, there are other types of conventional devices in which portions of high temperature in a picture around the vehicle captured by a pair of right and left stereo-cameras are recognized as objects, and the distance between the vehicle and the objects is calculated by obtaining the parallax of the objects so that targeting objects which may interfere with the running course of the vehicle can be detected based on a moving direction and a position of the objects, and an alarm can be output (refer to Japanese Unexamined Patent Application, First Publication No. 2001-6096, for example).
As mentioned above, if the extraction is carried out based only on the shape of the object as in the above-mentioned conventional device using the thresholding method, the shape of the targeting object becomes ambiguous due to the influence of a hat or clothing the targeting object (i.e., pedestrians, etc.) is wearing, and the environment surrounding the pedestrian. Also, when a vehicle is running, the height of a pedestrian is generally detected to be different from his/her real height due to the influence of change in the shape of a road ahead, or the pitting of the vehicle, and the barycentric coordinates of the target in the picture cannot be fixed with respect to the distance. Accordingly, it is difficult to stably carry out the extraction of only pedestrians.
For the reasons explained above, methods have been proposed in which only objects that are likely to be pedestrians are extracted by calculating the size of the objects in real space based on a gray scale image, and determining the positional relationship of the objects, images of which have been subjected to the thresholding method, or in which structures on the road and vehicles are extracted from objects subjected to the thresholding method, and they are recognized as objects other than pedestrians so as to be removed from the objects for which an alarm is necessary. However, using the above methods, when a vehicle is running in the rain, for instance, since objects are affected by the rain and the amount of infrared rays is changed, there is a problem that extraction of pedestrians cannot be stably carried out.
That is, in a typical environment, since the temperature of heat-retaining objects, which do not generate heat by themselves but retain heat which was externally applied, such as signboards, walls, and telegraph poles, decreases due to rain, it becomes difficult to detect them using an infrared camera (i.e., they tend not to appear in an infrared picture). Also, although heat generating objects which generate heat by themselves, such as vending machines, can be detected using an infrared camera, portions that emit infrared rays and can be detected are significantly decreased in the rain, and there is a tendency for an accurate determination of the shape thereof to become difficult.
Also, as for a pedestrian, although portions that are exposed to the outside (e.g., head) may be detected, portions of the pedestrian that are covered by the clothing may not be detected using an infrared camera depending on the degree to which the clothing is wet. That is, the situation around a vehicle differs even at the same point of a place depending on the weather, and the shape of all objects in a gray scale image detected by an infrared camera varies. Accordingly, there is a danger in the conventional methods that the extraction of only pedestrians from the image cannot be stably performed.
Moreover, when it is raining, the shape of objects in the image tends to become ambiguous since rain drops attach to the surface of the lens of a camera, and this makes the determination of the shape of the objects even more difficult.